54  External and Construct Validity

54.1 External validity

Overview

‘External validity’ refers to the extent to which the findings of a study can be generalised to settings, people, times, and measures other than those used in the original research.

It addresses the applicability of the research outcomes beyond the specific conditions under which the study was conducted.

  • High external validity means that the conclusions drawn from the study can be reasonably expected to hold true in other contexts, with different populations, and at different times.

  • It’s particularly significant in fields such as psychology, education, and health sciences, where researchers aim to apply their findings to real-world situations.

  • While internal validity focuses on the correctness of the study’s structure and execution, external validity centers on the broader applicability of its results.

Factors affecting external validity

Several factors can affect the external validity of a study:

  • Population validity reflects whether the study’s results can be generalised to a broader population beyond the sample used. This is influenced by how representative the sample is of the target population.

  • Ecological validity refers to the extent to which the study’s environment, procedures, and materials reflect the real-world settings to which the results will be applied.

  • Temporal validity is concerned with the time-related aspects of generalisability, testing whether the study findings will hold true at different times, under different conditions, or in different eras.

Challenges

Enhancing external validity presents unique challenges, particularly the balance between control (to ensure internal validity) and realism (to ensure external validity).

  • For example, highly controlled environments, often necessary for establishing cause-and-effect relationships, may lack the complexity and variability of real-world settings.

  • Additionally, achieving a truly representative sample can be difficult, especially when studying populations that are diverse or hard to access.

  • We must also consider the potential for changing norms and conditions over time, which can affect the applicability of our findings in the future.

Strategies

To enhance external validity, researchers can employ several strategies.

  • Using random sampling techniques to select participants increases the likelihood that the sample represents the broader population.

  • Conducting research in naturalistic settings, or using field experiments, can improve ecological validity by incorporating real-world complexities into the study.

  • Replicating studies across different populations, settings, and times is another effective approach, as it tests the consistency of the findings in varied conditions.

  • We can also use theoretical sampling, choosing samples based on specific theoretical criteria relevant to the research question, to explore the applicability of findings to different groups or situations.

  • Clearly defining the scope and limitations of the study’s applicability in the reporting phase helps in setting realistic expectations about the extent of generalisation possible from the research findings.

54.2 Construct validity

Overview

‘Construct validity’ is the extent to which a test or instrument measures the concept or construct it’s intended to measure.

Construct validity is about ensuring that the methods, tests, and procedures we use accurately capture the abstract concept being studied.

  • For example, in a study measuring intelligence, construct validity would involve examining whether the test truly measures intelligence, as theoretically defined, and not something else like memory or education level.

  • High construct validity means that the instrument is not only measuring the intended construct but is doing so in a way that is meaningful and theoretically aligned.

Components

Construct validity has several components, each addressing different aspects of how well a test measures a construct:

  • ‘Convergent’ validity refers to the degree to which a test correlates with other measures of the same construct, indicating that they are all measuring the same thing.

  • ‘Divergent’ or ‘discriminant’ validity, on the other hand, involves demonstrating that the test does not measure unrelated constructs. It’s confirmed when the test shows low correlation with measures of different constructs.

  • ‘Face’ validity, while more subjective, assesses whether the test appears to measure the intended construct at face value.

  • ‘Factorial’ validity, which involves using factor analysis, helps in understanding the underlying structure of a construct and ensuring that the test aligns with this structure. Factor analysis was previously covered here.

Challenges

Establishing construct validity can be challenging due to the abstract nature of many constructs.

  • Constructs like intelligence, motivation, or satisfaction are not directly observable and require indirect measurement through various indicators or behaviors.

  • Ensuring that these indicators truly represent the construct involves a deep understanding of the theoretical background and the conceptual definitions of the construct.

  • Constructs often evolve over time as research and theory develop, requiring continuous validation of measures.

  • There is potential overlap between constructs, making it difficult to achieve clear discriminant validity. We must carefully design our studies and select our instruments to adequately capture the complexity and nuances of the constructs under investigation.

Enhancing construct validity

  • To enhance construct validity, we can start with a thorough literature review to define the construct clearly and establish its theoretical underpinnings. Developing a strong theoretical framework is essential for guiding the selection or design of appropriate measures.

  • Employing multiple methods or measures to assess the construct, known as ’triangulation’, can also support construct validity. This might include a combination of surveys, interviews, observations, or experiments.

  • Statistical techniques like factor analysis are useful for examining the underlying structure of the construct and ensuring that the measurement aligns with this structure. Pilot testing and refinement of instruments based on feedback and initial results can further improve construct validity.

  • As noted previously, transparency in reporting the development, selection, and validation of measures, along with a critical discussion of their limitations, contributes to a better understanding and evaluation of construct validity in research.